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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed , 
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earned patent term adjustment. See 37 CFR 1.704(b). 
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2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 
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4) E3 Claim(s) 1^8 is/are pending in the application. 
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5) D Claim(s) is/are allowed. 

6) D Claim(s) is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 
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10)KI The drawing(s) filed on 24 March 2004 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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DETAILED ACTION 



Response to Amendment 

1 . This action is responsive to applicant's amendment and remarks received on 10/10/07. 
Claims 1-8 are currently pending. 



Specification 

2. In response to applicant's title amendment received on 10/10/07, the previous title 
objection is withdrawn. 



Claim Rejections - 35 USC § 102 
3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the applicant for a patent. 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 



4. Claims 1, 2, 3, 7, 8 are rejected under 35 U.S.C. 102(b) as being anticipated by Watanabe 
et al (EP 1043689 A2). 
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Regarding claim 1, Watanabe teaches an image processing device for determining three- 
dimensional position and/or orientation of an object, comprising: 

image data capturing means for capturing image data containing an image of the object 
(Watanabe: figure 1, numeral 20); 

model pattern creating means for creating a model pattern based on image data of a 
reference object with a reference orientation relative to said image capturing means captured by 
said image capturing means, said reference object having a shape substantially identical to that of 
the object (Watanabe: figure 5, numeral 102 the first model pattern is created when m equals 0); 

transformation means for performing two-dimensional and geometrical transformation of 
the created model pattern to generate a transformed model pattern representing an image of the 
object with an orientation different from the reference orientation (Watanabe: figure 5 a 
subsequent/second model pattern is created after m is incremented at figure 5, numeral 104 that 
has two-dimensional and geometrical transformation from figure 5, numeral 101); 

pattern matching means for performing a pattern matching of the image data of the object 
captured by said image capturing means with the transformed model pattern (Watanabe: figure 6, 
numeral 203); 

selecting means for repeatedly performing the generation of a transformed model pattern 
and the pattern matching of the image data of the object with the transformed model pattern to 
thereby select one of the transformed model patterns in conformity with the image data of the 
object, and obtain information on a position of the image of the object in the image data 
(Watanabe: figure 6, numerals 204-208); and 
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determining means for determining three-dimensional position and/or orientation of the 
object based on the information on the position of the image of the object in the image data and 
information on the orientation of the selected one of the transformed model patterns (Watanabe: 
figure 6, numeral 209). 

Regarding claim 2, Watanabe teaches an image processing device for determining three- 
dimensional position and/or orientation of an object, comprising: 

image data capturing means for capturing image data containing an image of the object 
(Watanabe: figure 1, numeral 20); 

model creating means for creating a model pattern based on image data of a reference 
object with a reference orientation relative to said image data, capturing means captured by said 
image data capturing means, said reference object having a shape substantially identical to that 
of the object (Watanabe: figure 5, numeral 102 the first model pattern is created when m equals 

0); 

. transformation means for performing two-dimensional and geometrical transformation of 
the created model pattern to generate a plurality of transformed model patterns each representing 
an image of the object with an orientation different from the reference position (Watanabe: figure 
5 a subsequent/second model pattern is created after m is incremented at figure 5, numeral 104 
that has two-dimensional and geometrical transformation from figure 5, numeral 101); 

storage means for storing the plurality of transformed model patterns and information on 
orientations of the respective transformed model patterns (Watanabe: figure 5, numeral 102, 
103); 
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pattern matching means for performing pattern matching of the image data of the object 
captured by said image capturing means with the plurality of transformed model patterns to 
thereby select one of the transformed model patterns in conformity with the image data of the 
object (Watanabe: figure 6, numerals 203-208), and obtain information on a position of the 
image of the object in the image data (Watanabe: figure 6, numerals 209); and 

determining means for determining three-dimensional position and/or orientation of the 
object based on information on the position of the image of the object in the image data and the 
information on an orientation of the selected one of the transformed model patterns (Watanabe: 
figure 6, numeral 209). 

Regarding claim 3, Watanabe teaches a two-dimensional and geometrical transformation 
of an affine transformation ("world coordinate system is obtained by an arithmetic operation of 
coordinate transformation using the data of the position/posture of the workpiece W in the 
camera coordinate system and the position/posture of the camera 20 in the world coordinate 
system"; Watanabe: paragraph [0042]), and said image processing device further comprises 
additional measuring means for obtaining a sign of inclination of the object with respect to said 
image capturing means ("position/posture of the workpiece in the camera coordinate system is 
stored as [xO, yO, zO, a0, p0, y0]c, where a, p, and y mean rotation angle around X-, Y-, Z- axes, 
and c c' means the camera coordinate system"; Watanabe: paragraph [0033]). 

Regarding claim 7, Watanabe teaches a storage means storing an operating orientation of 
the robot relative to the object or storing an operating orientation and an operating position of the 
robot relative to the object ("when a picking-up command is inputted .... three dimensional 
position/posture of the camera 20 on the world coordinate system at this image capturing 
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position is outputted to the image processing apparatus 30"; Watanabe: paragraph [0040]); and 
robot control means for determining an operating orientation of the robot or the operating 
orientation and an operating position of the robot based on the determined three-dimensional 
position and/or orientation of the object ("robot controller 10 operates the robot to perform a 
pick-up operation .... based on the three-dimensional position/posture of the workpiece W (Step 
210)"; Watanabe: paragraph [0044]). 

Regarding claim 8, Watanabe teaches an image capturing means mounted on the robot 
(Watanabe: figure 1, numeral 20). 

Claim Rejections - 35 (JSC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the combination 
of Watanabe et al (EP 1043689 A2) with Maeda et al (US 2003/0161537 Al), and further in 
view of Inoue (US 2003/0161504 Al). 

Regarding claim 4, Watanabe discloses all elements as mentioned above in claim 3. 
Watanabe does not teach dividing of a model pattern into at least two partial model patterns 
which are subject to the affine transformation to generate transformed partial model patterns, and 
pattern matching of the image data of the object with the transformed partial model patterns to 
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determine most conformable sizes, and determines the sign of the inclination based on 
comparison of the sizes of the conformable partial model patterns with each other. 

Maeda teaches a pattern matching of the image data of the object with the transformed 
model patterns to determine most conformable sizes, and determines the sign of the inclination 
based on comparison of the sizes of the conformable model patterns with each other (Maeda: 
figure 1, numeral 32). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe reference to determine the sign of the inclination based on 
comparison of the sizes of the conformable partial model patterns with each other as suggested 
by Maeda, to allow the imaging device to accurately identify u a three-dimensional object using 
images obtained by photographing the object in the various directions" (Maeda: paragraph 
[0004]). 

The Watanabe with Maeda combination as applied above does not teach dividing of a 
model pattern into at least two partial model patterns which are subject to the affine 
transformation to generate transformed partial model patterns. 

Inoue teaches dividing of a model pattern into at least two partial model patterns (Inoue: 
figure 2, numeral 302, 303), which are subject to the affine transformation ("affine 
transformation"; Inoue: paragraph [0107]) to generate transformed partial model patterns. 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe with Maeda combination as applied above, to utilize partial 
model patterns as suggested by Inoue, to further "correctly classify the input image regardless of 
a fluctuation in illumination, and a state of occlusion" (Inoue: paragraph [0012]). 
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7. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Watanabe et al 
(EP 1043689 A2) in view of Okisu et al (US 6,806,903 Bl). 

Regarding claim 5, Watanabe discloses all elements as mentioned above in claim 3. 
Watanabe does not teach a measurement of distances from a displacement sensor separately 
provided in the vicinity of said image capturing means to at least two points on the object using 
the displacement sensor, and determines the sign of the inclination based on comparison of the 
measured distances. 

Okisu teaches a measurement of distances from a displacement sensor separately 
provided in the vicinity of said image capturing means to at least two points on the object using 
the displacement sensor, and determines the sign of the inclination based on comparison of the 
measured distances ("measure distance to two separate portions of an object from the electronic 
camera, and calculate an angle of inclination 0 based on measure two distances"; Okisu: col. 19, 
lines 54-57). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe reference to determine the sign of the inclination based on 
comparison of the measured distances as suggested by Okisu, to allow the image processing 
device to properly focus with high degree of accuracy on the targeted object for image capturing. 

8. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Watanabe et al 
(EP 1043689 A2) in view of Maeda et al (US 2003/0161537 Al). 

Regarding claim 6, Watanabe discloses all elements as mentioned above in claim 3. 
Watanabe further teaches an additional pattern matching of image data of the object captured 
after said image data capturing means is slightly moved or inclined ("camera may be moved 
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parallelly in accordance with the position of the workpiece in the field of view of the camera"; 
Watanabe: paragraph: [0048]). Watanabe does not teach determining the sign of the inclination 
based on judgment whether an inclination of image of the object becomes larger or smaller than 
the selected one of the transformed model patterns. 

Maeda teaches determining the sign of the inclination based on judgment whether an 
inclination of image of the object becomes larger or smaller than the selected one of the 
transformed model patterns (Maeda: figure 1, numeral 32). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe reference to determine the sign of the inclination based on 
judgment whether an inclination of image of the object becomes larger or smaller than the 
selected one of the transformed model patterns as suggested by Maeda, to allow the imaging 
device to accurately identify "a three-dimensional object using images obtained by 
photographing the object in the various directions" (Maeda: paragraph [0004]). 

Response to Arguments 
9. Applicant's arguments filed on 10/10/07 with respect to claim 1, have been fully 
considered but they are not persuasive. Applicant argues that Watanabe does not disclose two- 
dimensional and geometrical transformation representing an image of the object with an 
orientation different from a reference orientation. This argument is not considered persuasive 
since Watanabe does meet the limitations of claim 1. In figure 5, Watanabe takes a first image 
when m=0 and this can be considered the first model pattern. The next image taken of the object 
at m=l can be considered the transformed model pattern. The claim language states that 
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"transformation is performed of a created model pattern", this implies that the two-dimensional 
and geometrical transformation does not have to be on the model pattern that is captured by the 
camera/model pattern creating means. The claim language, "transformation is performed of a 
created model pattern", can be interpreted as creating a transformed model pattern that represents 
the reference object. This interpretation is met by figure 5, numeral 101 when m=l, since the 
camera is repositioned which changes the two-dimensional and geometrical transformation, 
captures a new image data, and produces a "transformed model pattern". 

In regards to claim 2, applicant argues that Watanabe does not disclose pattern matching 
means for performing pattern matching of the image data of the object captured by said image 
capturing means with the plurality of transformed model patterns. This argument is not 
considered persuasive since Watanabe meets the limitations of the claim and can be seen in the 
rejection above. 

In regards to claims 3-8, applicant argues that the claims distinguish over Watanabe due 
to the prior art failing to make up for the deficiencies in claim 1 . This argument is not 
considered persuasive and the rejection of claims 3-8 can be seen above. 

Conclusion 

10. THIS ACTION IS MADE FINAL, Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
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the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Edward Park whose telephone number is (571) 270-1576. The 
examiner can normally be reached on M-F 10:30 - 20:00, (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian Werner can be reached on (571) 272-7401 . The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Edward Park 
Examiner 
Art Unit 2624 
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